Building Infrastructure Using Azure Resource Manager

Cloud APIs are bifurcated into two categories, the management plane and the data plane. The management plane is an API that controls the creation, deletion, and mutation of infrastructure. The data plane is an API exposed by provisioned infrastructure.

For example, the management plane would be used to create a SQL database. The data plane for the SQL database resource would be the SQL protocol for manipulating data and structure within the database.

The management plane is serviced by the cloud resource API, and the data plane is serviced by the API exposed by the provisioned service.

In this lesson, we will learn how to use the Azure SDK for Go to provision infrastructure in Azure. We will learn how to create and destroy resource groups, virtual networks, subnets, public IPs, virtual machines, and databases. The goal of this section is to build awareness of the Azure Go SDK and how to interact with ARM.

Azure SDK for Go#

Cloud SDKs simplify the interaction between a given language and a cloud provider's API. In the case of Azure, we will be using the Azure SDK for Go to interact with the Azure APIs. Specifically, we'll use the latest edition of the SDK, which has been redesigned to follow the Azure design guidelines for Go. For the latest information about packages and docs, be sure to check out the Azure SDK Releases page.

The code for this section is located here on GitHub.

Setting up the environment#

To run the code for this section, we will need to set up a Secure Shell (SSH) key and an .env file.

Use the terminal above for the commands below:

This command will create a .ssh directory, generate an SSH key pair within the directory, and ensure that proper permissions are set on the key pair.

Note: The preceding command creates an SSH key that does not have a passphrase. We are only using this key pair as an example. We should provide a strong passphrase for real-world usage.

Next, let's set up a local .env file that we will use to store environmental variables used in the examples:

Now, this command will create an .env file that contains two environment variables, AZURE_SUBSCRIPTION_ID and SSH_PUBLIC_KEY_PATH. We derive the value for the Azure subscription ID from the Azure CLI's current active subscription.

Now that we have set up our local environment let's build an Azure virtual machine that will run a cloud-init provisioning script and provide access using SSH via a public IP.

Building an Azure virtual machine#

Let's get started by running the example, and then we'll delve into the code for building the infrastructure.

To run the example, run the following command in the terminal above:

After running go run ./cmd/compute/main.go, the output shows that the program built several bits of infrastructure, including an Azure resource group, network security group, virtual network, and virtual machine. We'll discuss every piece of infrastructure in more detail soon.

As the output states, we can also use SSH to access the virtual machine, as described in the output. We'll use this to explore the provisioned state of the virtual machine to confirm that the cloud-init provisioning script ran as expected.

If we visit the Azure portal, we should see the following:

In the preceding screenshot, we can see the resource group (billowing-resonance) as well as all of the infrastructure created. Next, let's look at the code that provisioned this infrastructure.

Provisioning Azure infrastructure using Go#

In these examples, we will see how to build Azure API clients, probe for credentials for accessing APIs, and mutate infrastructure. Many of these examples use abbreviated error-handling behavior to keep the code as concise as possible for illustrative purposes. panic is not your friend. Please wrap and bubble your errors as appropriate.

Let's start with the entry point of go run ./cmd/compute/main.go and learn how to use Go to provision cloud infrastructure:

In the preceding code, we load environment variables in the local .env file using godotenv.Load(). In main, we create a new VirtualMachineFactory to manage the creation and deletion of Azure infrastructure. Once the infrastructure is created in factory.CreateVirtualMachineStack, we print the SSH connection details and prompt for user confirmation to delete the infrastructure stack.

Next, let's dive into the VM factory and see what is included in the VM stack:

This code defines the structure of VirtualMachineFactory, which is responsible for the creation of and access to the Azure SDK API clients. We instantiate those clients using the NewVirtualMachineFactory func, as shown here:

This code builds a new default Azure identity credential. This credential is used to authenticate the client to the Azure APIs. By default, this credential will probe multiple sources for an identity to use. The default credential will probe for environment variables first, then it will attempt to use an Azure managed identity, and finally, it will fall back to using the Azure CLI's user identity. For this example, we are relying on the Azure CLI identity to interact with the Azure APIs. This is convenient for development but should not be used for a deployed application or script. Non-interactive authentication requires either an Azure service principal or an Azure-managed identity.

The VM factory builds each of the Azure API clients using subscriptionID, the credential, and the New* function for each of the clients. BuildClient() builds each client.

Now that we know how credentials and the API clients are instantiated let's dive into the creation of infrastructure in CreateVirtualMachineStack:

In the preceding code, we created the idea of a stack – a collection of related infrastructure. We created a new stack with a given location, a human-readable name, and the contents of the SSH public key path. Subsequently, we created each of the Azure resources needed to create a VM with public SSH access.

Let's explore each of the create and get funcs in CreateVirtualMachineStack:

In the preceding code, createResourceGroup calls CreateOrUpdate on groupsClient to create an Azure resource group in the specified location. An Azure resource group is a logical container for Azure resources. We will use the resource group as a container for the rest of our resources.

Next, let's dive into the network security group creation function, createSecurityGroup:

In the preceding code, we built an Azure network security group, which contains a single security rule to allow network traffic on port 22, enabling SSH access for the VM. Note that rather than calling CreateOrUpdate, we call BeginCreateOrUpdate, which issues PUT or PATCH to the Azure API and starts a long-running operation.

A long-running operation in Azure is one that—once the initial mutation is accepted—executes until it reaches a terminal state. For example, when creating a network security group, the API receives the initial mutation and then starts to build the infrastructure. After the infrastructure is ready, the API will indicate it is completed through the operation state or the provisioning state. poller takes care of following the long-running operation to completion. In HandleErrPoller, we follow the polling to completion and return the final state of the resource.

Next, let's explore the creation of the virtual network via createVirtualNetwork:

In the previous code block, we built an Azure virtual network for our VM. The virtual network is set up with a 10.0.0.0/16 Classless Inter-Domain Routing (CIDR) and a single subnet with a 10.0.0.0/24 CIDR. The subnet references the network security group we built in the previous code block, which causes the rules in the network security group to be enforced on the subnet.

Now that we have built the networking for our VM, let's build it via createVirtualMachine:

There is not much to show for createVirtualMachine(). As we can see, the same pattern of resource creation through a long-running API invocation is applied in this code. The interesting bits are in linuxVM():

In linuxVM, we specify the location, name, and properties of the VM. In the properties, we specify the type of hardware we'd like to provision. In this case, we are provisioning a Standard D3v2 (we can read more about it here on Microsoft's website) hardware Stock-Keeping Unit (SKU).

We also specify our StorageProfile, which is used to specify the OS as well as the data disks we'd like attached to the VM. In this case, we specify that we'd like to run the latest version of Ubuntu 18.04. Both NetworkProfile and OSProfile are a little too complex to include in this function, so let's explore them individually in the following code block:

In networkProfile(), we create NetworkProfile, which specifies that the VM should have a single network interface using IPv4 and be exposed via a public IP. The network interface should be allocated on the subnet that we created in createVirtualNetwork().

Next, let's explore the OSProfile configuration via linuxOSProfile() in the following code block:

In linuxOSProfile, we create an OSProfile, which includes details such as the admin username, computer name, and SSH configuration. Take note of the CustomData field used for specifying the Base64-encoded cloud-init YAML, which is used to run the initial configuration of the VM.

Let's explore what we are doing in the cloud-init YAML:

Once the VM is created, the following cloud-init instructions are executed:

• Line 2: First, the packages on the Ubuntu machine are upgraded.

• Lines 3–5: Next, the nginx and golang packages are installed via the Advanced Package Tool (APT).

• Lines 6–7: Finally, runcmd echos "hello world".

cloud-init is super-useful for bootstrapping VMs. If you have not used it previously, we highly recommend exploring it further.

We can verify cloud-init executed by accessing the VM using SSH and executing commands similar to the following. Remember, your IP address will be different than what is shown here:

As we can see, nginx and go have been installed. We should also see the APT mutations and hello world in /var/log/cloud-init-output.log on the provisioned VM.

We have provisioned and created an Azure VM and related infrastructure! Now, let's destroy the entire stack of infrastructure. We should be able to press “Enter” in the shell where we are running go run ./cmd/compute/main.go.

Let's see what happened when we called factory.DestroyVirtualMachineStack:

In DestroyVirtualMachineStack, we simply call BeginDelete() on the group's client, specifying the resource group name. However, unlike other examples, we do not wait for the poller to complete. We send the DELETE HTTP request to Azure. We do not wait for the infrastructure to be completely deleted; instead, we trust that the acceptance of delete means that it will eventually reach the deleted terminal state.

We have now built and cleaned up a stack of infrastructure using the Azure SDK for Go. We have learned how to create resource groups, virtual networks, subnets, public IPs, and VMs, and how a pattern can be extended to any resource in Azure. Additionally, these skills are applicable to each of the major clouds, not just Azure. AWS and GCP both have similar concepts and API access patterns.